| 0 | 1 | 2 | 3 | 4 | 5 | |
|---|---|---|---|---|---|---|
| 0 | -0.409626 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 1 | -0.409626 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | -0.409626 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | -0.409626 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 4 | -0.409626 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
TP-101b motion profile
Large Amplitude Motion Platform (LAMP) Characterization Dataset Description
National Renewable Energy Laboratory (NREL) Water Power Technology Validation
1 Overview
The Large Amplitude Motion Platform (LAMP) at the National Renewable Energy Laboratory (NREL) Water Power Technology Validation Facility is a six-degree-of-freedom (6-DOF) motion platform intended to be used for dry testing of wave energy converters (WECs) and other marine energy devices.
This MHKDR submission contains data collected during an initial characterization of the LAMP. The intent of this characterization was to run the LAMP through a series of gradually more complex motion profiles to understand how the LAMP performs when it is push to and beyond its limits specified by the manufacturer. The data collected during this characterization is provided in this submission along with documentation of the provided data. This dataset is intended to be used by researchers and developers working on marine energy devices to prepare for testing on the LAMP by understanding the required LAMP data inputs and expected outputs.
The data generated during this characterization was collected using the LAMP built-in data acquisition system. The data was originally recorded in a proprietary binary format and has been converted to Parquet format for ease of use. The data has been filtered to only include 6-DOF motion inputs and outputs.
2 Dataset Contents
This MHKDR submission contains two distinct datasets.
The “Command” datasets contains the 6-DOF input motion profiles that are passed into the LAMP. The LAMP requires 6DOF position, (meters/degrees) timeseries. Commanded datasets are provided in both CSV format and include a metadata json file describing the specification of the test profiles. The command data file structure is described in Section 3.1, and the dataset is described in Section 4.1.
The “Response” datasets contains the 6-DOF output of the LAMP. The LAMP outputs both the input and output position, velocity, and acceleration for each of the 6 degrees of freedom. The response represents the calculated motion of the platform based on the measured actuator positions. The response data file structure is described in Section 3.2, and the dataset is described in Section 4.2.
3 File Structure
3.1 Command
Data within the nrel_lamp_characterization_command.zip file contains the commanded 6-DOF motion profiles that generated as input to the LAMP. Each subdirectory within nrel_lamp_characterization_command/ contains a different motion profile
▾ nrel_lamp_characterization_command/
▸ TP-101/
▸ TP-101a/
▸ TP-101b/
▸ TP-102/
▸ TP-103/
▸ TP-104/
▸ TP-105/
...command test profiles.
Within a single motion profile the data is organized as follows
▾ TP-101b/
▾ csv/
TP-101b_Acceleration.csv
TP-101b_Position.csv
TP-101b_Time.csv
TP-101b_Velocity.csv
▾ viz/
TP-101b_Surge.png
TP-101b_metadata.jsonSection 4.1 has details about the contents of the command dataset files.
3.2 Response
Data within the response/ directory contains the measured 6-DOF motion outputs from the LAMP.
▾ response/
▾ Char_001.NREL_LAMP_Char.TP101a-using-TP101a.one_dof_hexframe.a1.20240628.090000/
Char_001.NREL_LAMP_Char.TP101a-using-TP101a.one_dof_hexframe.a1.20240628.090000.parquet
▸ Char_002.NREL_LAMP_Char.RW101b-using-TP101b.one_dof_hexframe.a1.20240708.092000/
▸ Char_003.NREL_LAMP_Char.RW102-using-TP102.one_dof_hexframe.a1.20240708.103200/
▸ Char_004.NREL_LAMP_Char.RW103-using-TP103.one_dof_hexframe.a1.20240708.113500/
▸ Char_005.NREL_LAMP_Char.RW104-using-TP104.one_dof_hexframe.a1.20240708.122300/Response files use the following naming convention:
Char_<ID>.NREL_LAMP_Char.<TestName>-using-<CommandProfile>.<LAMP Configuration>.<Data Level>.<Date>.<Time>.parquetWhere:
Char_<ID>is a sequential unique identifier for each testNREL_LAMP_Charidentifies the LAMP characterization dataset<TestName>-using-<MotionProfile>is the name of the test and the name of command profile used for that test<LAMP Configuration>identifies the configuration of the test setup / LAMP configuration used for the this test- Valid values for LAMP characterization tests include:
<number of degrees of freedom specified in motion>-<weight/frame setup>:- Where
<number of degrees of freedom specified in motion>is one of:one_dof: single degree of freedom motion profiletwo_dof: two degree of freedom motion profilethree_dof: three degree of freedom motion profilesandia_multisine: six degree of freedom motion profile
- And
<weight/frame setup>is one of:hexframe: LAMP with hexagonal frame onlymass_1-1774_lbs: LAMP with hexagonal frame and 1774 lbs (804.67 kg) test mass attachedmass_2-2309_lbs: LAMP with hexagonal frame and 2309 lb (1,047.34 kg) test mass attached
- Where
- Valid values for LAMP characterization tests include:
Section 4.1 has details about the contents of the command data files.
4 Channel Definitions
4.1 Command Dataset
CSV files contain the commanded timeseries data for each of the 6 degrees of freedom using the order Surge, Sway, Heave, Roll, Pitch, Yaw. Units for Surge, Sway, and Heave are in meters [m] and units for Roll, Pitch, and Yaw are in degrees [deg]. Time is provided in seconds [s]. Each CSV file does not contain any header or an index column:
The test profiles generated for LAMP characterization are described in a JSON file. These are specific to each motion profile and contain the specification for the motion profile generation. These are not necessary for testing on the LAMP, but are provided as an example of how a user can generate a file that defines some values of interest for a motion profile.
command_json = json.load(open(Path(command_path, f"{profile}_metadata.json")))
print(json.dumps(command_json, indent=2)){
"spec": {
"name": "TP-101b",
"inputs": {
"Surge": {
"number_of_iterations": 10,
"individual_test_duration_seconds": 180,
"min_frequency_start_hz": 0.01,
"min_frequency_end_hz": 0.01,
"max_frequency_start_hz": 0.5,
"max_frequency_end_hz": 0.2,
"min_amplitude_start": 0.1,
"min_amplitude_end": 1.8,
"max_amplitude_start": 0.1,
"max_amplitude_end": 1.8,
"units": "meters"
}
},
"is_locked": true,
"detail": null
},
"Surge": {
"test_start_indexes": [
1000,
21000,
41000,
61000,
81000,
101000,
121000,
141000,
161000,
181000
],
"test_end_indexes": [
19000,
39000,
59000,
79000,
99000,
119000,
139000,
159000,
179000,
199000
],
"max_accelerations": [
0.9867375279566659,
2.483304787374019,
3.541714063295676,
4.21165694140524,
4.542788610948367,
4.585231002606216,
4.3890739136542845,
4.004882164622137,
3.4843691123992437,
2.880450443281113
],
"end_frequencies": [
0.5,
0.4666666666666667,
0.43333333333333335,
0.4,
0.3666666666666667,
0.33333333333333337,
0.3,
0.26666666666666666,
0.23333333333333334,
0.2
]
}
}TP-101b motion profile
For example we can now use this specification to visualize surge position at iteration 1:
target_iteration = 1
target_index = target_iteration - 1 # zero indexed
surge_start_index = command_json["Surge"]["test_start_indexes"][target_index]
surge_end_index = command_json["Surge"]["test_end_indexes"][target_index]
position_df[0].iloc[surge_start_index:surge_end_index].plot(
title=f"{profile} Commanded Surge Position, Iteration {target_iteration}",
ylabel="Surge Position [m]",
xlabel="Sample Index",
figsize=(10, 3),
)
Additionally, each test profile contains a set of PNG visualizations of each degree of freedom that is specified to be moving, for this test profile this is viz/TP-101b_Surge.png.
4.2 Response Dataset
A single response dataset contains the motion outputs from the LAMP. Data is stored in .parquet files, an open source columnar storage file format that is optimized efficient data storage and retrieval. At the time of publishing this dataset there are many ways to read parquet files including in Python using Pandas or Polars, MATLAB, R, Julia, or DuckDB.
In this documentation we will show examples of reading the parquet files using Pandas in Python and parquetread in MATLAB.
4.2.1 Reading Parquet Files with Python or MATLAB
In Python the Pandas pandas package is used to read a parquet file and inspect the data types of each column:
import pandas as pd
df = pd.read_parquet("path_to_parquet_file.parquet")
df.info()In MATLAB can use parquetread to read a parquet file and inspect the data types of each column.
df = parquetread('path_to_parquet_file.parquet');
summary(df)4.2.2 Response Dataset Description
There are three distinct types of columns in the LAMP dataset, time, command 6DOF PVA and response 6DOF PVA. Time columns use time provided by the LAMP control system. For this dataset sampling frequency is 100HZ for test cases up to Char_023, and 200Hz for test cases Char_024 and beyond. Command columns represent the input motion profiles prepared by the LAMP system. Response columns represent the measured 6 dof position of the LAMP based on actuator positions. The LAMP data nomenclature for input 6DOF is KinematicsFwd and output or response 6DOF is KinematicsBwd.
| Loading ITables v2.4.4 from the internet... (need help?) |
4.2.3 Response Dataset Quality Control
An additional quality control (QC) step was performed to flag any data quality issues. This QC step includes an additional qc_<column_name> column after each data column that contains a bitflag that represents the results of a series of quality checks. Each bit in the bitflag represents a specific quality check, where a value of 0 indicates the check passed and a value of 1 indicates the check failed.
A QC value of 0 indicates valid data. Any value greater than 0 indicates at least one QC failure.
These QC flags are intended to provide an indication of the quality of the underlying data. This dataset is relatively high quality because the laboratory testing environment is controlled and reliable, but the flags are provided for completeness and to facilitate automated data quality assessment.
4.2.3.1 QC Bitflag Specification
| Bit | Check Name | Type | Description |
|---|---|---|---|
| 0 | time_not_monotonic |
Dataset-level | 1 if time is NOT monotonically increasing |
| 1 | value_below_min |
Row-level | 1 if value < QC_Min |
| 2 | value_above_max |
Row-level | 1 if value > QC_Max |
| 3 | value_missing |
Row-level | 1 if value is NA/null |
| 4 | value_stagnant |
Dataset-level | 1 if column std() == 0 |
Example QC values and their meanings:
0,0b00000: All checks passed - valid data1,0b00001: Bit 0 is set - Time is not monotonic increasing, time is invalid and all other data is may not be valid2,0b00010: Bit 1 set - instantaneous value below QC minimum, data is likely invalid (noise, measurement error, etc)4,0b00100: Bit 2 set - instantaneous value above QC maximum, data is likely invalid (noise, measurement error, etc)8,0b01000: Bit 3 set - instantaneous value is missing (NA)16,0b10000: Bit 4 set - column has no variation (stagnant), data is likely invalid (stuck sensor, disconnected, etc)
QC values can be combined by summing the individual bit values. For example:
3,0b00011: Bit 0 and 1 is set - Time is not monotonic increasing and instantaneous value below QC minimum
These QC flags can be used to filter or clean the data as needed. The following sections show examples on how to use the QC flags.
4.2.3.2 Using LAMP characterization QC values in Python
4.2.3.2.1 Calculating Overall QC Statistics Using QC Flags
import pandas as pd
# Read parquet file with QC columns
df = pd.read_parquet("path_to_file.parquet")
# Calculate overall QC statistics (total failures in dataset)
qc_columns = [col for col in df.columns if col.startswith('qc_')]
total_failures = (df[qc_columns] > 0).sum().sum()
total_datapoints = len(df) * len(qc_columns)
valid_percentage = ((total_datapoints - total_failures) / total_datapoints) * 100
print(f"Total failures: {total_failures}/{total_datapoints}")
print(f"Valid data: {valid_percentage:.2f}%")4.2.3.2.2 Calculating Per Channel QC Statistics Using QC Flags
for qc_col in qc_columns:
failure_count = (df[qc_col] > 0).sum()
print(f"{qc_col}: {failure_count}/{len(df)} failures")4.2.3.2.3 Converting/Filtering Flagged QC values
# Example 1: Convert QC failures to NaN
for col in df.columns:
if not col.startswith('qc_'):
qc_col = f'qc_{col}'
if qc_col in df.columns:
df.loc[df[qc_col] > 0, col] = pd.NA
# Example 2: Filter rows that fail any QC check
df_clean = df[(df[qc_columns] == 0).all(axis=1)]
# Example 3: Replace QC failures with column mean
for col in df.columns:
if not col.startswith('qc_'):
qc_col = f'qc_{col}'
if qc_col in df.columns:
col_mean = df.loc[df[qc_col] == 0, col].mean()
df.loc[df[qc_col] > 0, col] = col_mean4.2.3.3 MATLAB QC Examples
4.2.3.3.1 Calculating Overall QC Statistics Using QC Flags
% Read parquet file with QC columns
df = parquetread('path_to_file.parquet');
% Calculate overall QC statistics (total failures in dataset)
qc_cols = startsWith(df.Properties.VariableNames, 'qc_');
qc_data = df{:, qc_cols};
total_failures = sum(qc_data(:) > 0);
total_datapoints = numel(qc_data);
valid_percentage = ((total_datapoints - total_failures) / total_datapoints) * 100;
fprintf('Total failures: %d/%d\n', total_failures, total_datapoints);
fprintf('Valid data: %.2f%%\n', valid_percentage);4.2.3.3.2 Calculating Per Channel QC Statistics Using QC Flags
% Calculate per-channel QC failure counts
qc_names = df.Properties.VariableNames(qc_cols);
for i = 1:length(qc_names)
failure_count = sum(df.(qc_names{i}) > 0);
fprintf('%s: %d/%d failures\n', qc_names{i}, failure_count, height(df));
end4.2.3.3.3 Converting/Filtering Flagged QC values
% Example 1: Convert QC failures to NaN
data_cols = ~startsWith(df.Properties.VariableNames, 'qc_');
for i = find(data_cols)
col_name = df.Properties.VariableNames{i};
qc_col_name = ['qc_' col_name];
if ismember(qc_col_name, df.Properties.VariableNames)
df.(col_name)(df.(qc_col_name) > 0) = NaN;
end
end
% Example 2: Filter rows that fail any QC check
valid_rows = all(qc_data == 0, 2);
df_clean = df(valid_rows, :);
% Example 3: Replace QC failures with column mean
for i = find(data_cols)
col_name = df.Properties.VariableNames{i};
qc_col_name = ['qc_' col_name];
if ismember(qc_col_name, df.Properties.VariableNames)
valid_mask = df.(qc_col_name) == 0;
col_mean = mean(df.(col_name)(valid_mask), 'omitnan');
df.(col_name)(~valid_mask) = col_mean;
end
end